Papers with Transformer models
RTSM: Knowledge Distillation with Diverse Signals for Efficient Real-Time Semantic Matching in E-Commerce (2025.naacl-industry)
Copied to clipboard
| Challenge: | e-commerce product search is a key component of product discovery and sales in ecommerce . high computational demands of large transformer models pose challenges for their deployment in real-time scenarios. |
| Approach: | They propose a framework for real-time semantic matching that leverages both soft labels from a teacher model and ground truth generated from pairwise query-product and query-query signals. |
| Outcome: | The proposed framework outperforms teacher models and state-of-the-art models on e-commerce datasets. |
Making Asynchronous Stochastic Gradient Descent Work for Transformers (D19-56)
Copied to clipboard
| Challenge: | Asynchronous stochastic gradient descent (SGD) converges poorly for Transformer models . synchronous SGD is faster at raw training speed since it avoids waiting for synchronization . |
| Approach: | They propose a method to restore convergence by summing several asynchronous updates instead of applying them immediately. |
| Outcome: | The proposed method achieves the same BLEU score 1.36 times faster than asynchronous SGD. |
LightSeq: A High Performance Inference Library for Transformers (2021.naacl-industry)
Copied to clipboard
| Challenge: | Existing inference frameworks for natural language processing are not the best choice for online service of sequence processing problems. |
| Approach: | They propose a highly efficient inference library for Transformer models that includes GPU optimization techniques to streamline computation and reduce memory footprint. |
| Outcome: | The proposed library achieves 14x speedup compared with TensorFlow and 1.4x speed up compared to a concurrent CUDA implementation. |
Multimodal Graph Transformer for Multimodal Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | a myriad of complex tasks require both prior knowledge and reasoning intelligence. |
| Approach: | They propose a plug-and-play quasi-attention mechanism to integrate multimodal graph information to vanilla self-attention as effective prior. |
| Outcome: | The proposed model is able to perform reasoning across multiple modalities. |
ETC: Encoding Long and Structured Inputs in Transformers (2020.emnlp-main)
Copied to clipboard
Joshua Ainslie, Santiago Ontanon, Chris Alberti, Vaclav Cvicek, Zachary Fisher, Philip Pham, Anirudh Ravula, Sumit Sanghai, Qifan Wang, Li Yang
| Challenge: | Existing models for natural language processing (NLP) have been challenging to scale attention to longer inputs. |
| Approach: | They propose an extended Transformer construction architecture that scales attention to longer inputs by combining global-local attention with relative position encodings and a "Contrastive Predictive Coding" objective. |
| Outcome: | The proposed architecture scales attention to longer inputs and encodes structured inputs. |
Sarah’s Participation in WAT 2019 (D19-52)
Copied to clipboard
| Challenge: | Using the Transformer architecture, we trained similar systems across different tasks. |
| Approach: | They presented their results in the 6th Workshop on Asian Translation (WAT) translation task and their submissions to the task. |
| Outcome: | The proposed models perform better on tasks with smaller datasets and with smaller heads on multilingual datasets. |
Using Neural Machine Translation Methods for Sign Language Translation (2022.acl-srw)
Copied to clipboard
| Challenge: | Sign languages are the main medium of exchanging information for the deaf and hard of hearing. |
| Approach: | They propose to use two NMT architectures to train models on parallel German Sign Language corpora . they achieve substantial improvement in BLEU scores for the models trained on the two corporales . |
| Outcome: | The proposed models achieve significant improvements on the two corpora trained on the german sign language . the proposed models outperform the models trained on both corporales . |
VOLTA: Improving Generative Diversity by Variational Mutual Information Maximizing Autoencoder (2024.findings-naacl)
Copied to clipboard
| Challenge: | generative diversity is a critical yet underexplored issue in natural language generation . previous approaches to enhance diversity of Transformer models have been limited by their latent variables . |
| Approach: | They propose a framework that bridges Transformer with VAE to enhance generative diversity. |
| Outcome: | The proposed framework improves generative diversity while maintaining generative quality. |
Fusing Temporal Graphs into Transformers for Time-Sensitive Question Answering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for extracting temporal information from text are not suitable for time-sensitive questions. |
| Approach: | They propose to use existing temporal information extraction systems to construct temporal graphs of events, times, and temporal relations in questions and documents. |
| Outcome: | The proposed method outperforms graph convolution-based approaches on SituatedQA and TimeQA. |
Learning Adaptive Axis Attentions in Fine-tuning: Beyond Fixed Sparse Attention Patterns (2022.findings-acl)
Copied to clipboard
Zihan Wang, Jiuxiang Gu, Jason Kuen, Handong Zhao, Vlad Morariu, Ruiyi Zhang, Ani Nenkova, Tong Sun, Jingbo Shang
| Challenge: | Adaptive Axis Attention learns different attention patterns for each task and model layer . sparse attention patterns do not improve the run time of the models but they reduce model memory requirements . |
| Approach: | They propose a method that learns different attention patterns for each Transformer layer . they propose 'adaptive axis attention' method that identifies important tokens . |
| Outcome: | The proposed method does not require pre-training to accommodate sparse attention patterns. |
Multi-Unit Transformers for Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results show that the MUTE models outperform the Transformer-Base by up to +1.52, +1.99 and +1.00 BLEU points, with only a mild drop in inference speed (about 3.1%). |
| Approach: | They propose to use multiple parallel units to promote the expressiveness of the Transformer by introducing diverse and complementary units. |
| Outcome: | The proposed models outperform the Transformer-Base model with only a mild drop in inference speed (about 3.1%). |
An Evaluation of Neural Machine Translation Models on Historical Spelling Normalization (C18-1)
Copied to clipboard
| Challenge: | In this paper, we apply different NMT models to the problem of historical spelling normalization for five languages . we find that NMT model is much better than SMT in terms of character error rate . |
| Approach: | They propose to use NMT models to solve the problem of historical spelling normalization in five languages. |
| Outcome: | The proposed method improves historical spelling normalization for five languages. |
Hierarchical Transformers Are More Efficient Language Models (2022.findings-naacl)
Copied to clipboard
Piotr Nawrot, Szymon Tworkowski, Michał Tyrolski, Lukasz Kaiser, Yuhuai Wu, Christian Szegedy, Henryk Michalewski
| Challenge: | Transformers are impressive but inefficient and costly, which limits their applications and accessibility. |
| Approach: | They first use different ways to downsample and upsamplify activations in Transformers to make them hierarchical. |
| Outcome: | The proposed model outperforms Transformers on the ImageNet32 and enwik8 benchmarks. |
AMRs Assemble! Learning to Ensemble with Autoregressive Models for AMR Parsing (2023.acl-short)
Copied to clipboard
| Challenge: | et al., 2013) examines the current state-of-the-art in AMR parsing . current models violate structural constraints, but they can corrupt graphs . |
| Approach: | They propose two new ensemble strategies to improve AMR parsing robustness and reduce computational time. |
| Outcome: | The proposed methods improve robustness to structural constraints while reducing computational time. |
Transfer Learning from Transformers to Fake News Challenge Stance Detection (FNC-1) Task (2020.lrec-1)
Copied to clipboard
| Challenge: | In the last two years, significant improvements have occurred in NLP with the development of large language models using contextualized word embeddings based on the Google Transformer architecture. |
| Approach: | They performed experiments on data from the Fake News Challenge stage 1 (FNC-1) they used BERT sentence embeddings as a model feature and BERT, XLNet, and RoBERTa transformers to fine-tune them. |
| Outcome: | The proposed model outperforms the winner's system on class-wise F1 scores and achieves state-of-the-art on the stance detection task. |
IrEne: Interpretable Energy Prediction for Transformers (2021.acl-long)
Copied to clipboard
| Challenge: | Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. |
| Approach: | They propose an interpretable and extensible energy prediction system that predicts inference energy consumption of Transformer-based NLP models. |
| Outcome: | The proposed system predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. |
BanglaRQA: A Benchmark Dataset for Under-resourced Bangla Language Reading Comprehension-based Question Answering with Diverse Question-Answer Types (2022.findings-emnlp)
Copied to clipboard
Syed Mohammed Sartaj Ekram, Adham Arik Rahman, Md. Sajid Altaf, Mohammed Saidul Islam, Mehrab Mustafy Rahman, Md Mezbaur Rahman, Md Azam Hossain, Abu Raihan Mostofa Kamal
| Challenge: | a lack of diverse and comprehensive question-answering datasets exists in under-resourced languages like Bangla. |
| Approach: | They propose a reading comprehension-based Bangla question-answering dataset . the dataset includes answerable and unanswerable questions covering four categories of questions . |
| Outcome: | The proposed dataset shows that it performs well as a training resource in high-resource languages. |
A Simple and Effective Positional Encoding for Transformers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies suggest that relative position encodings provide better performance than absolute position coding. |
| Approach: | They propose a mechanism to encode position and segment information into Transformer models using relative position encodings. |
| Outcome: | The proposed method achieves faster training and inference time while achieving competitive performance on GLUE, XTREME and WMT benchmarks. |
News Headline Grouping as a Challenging NLU Task (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in Natural Language Understanding (NLU) have seen models outperform human performance on many standard tasks. |
| Approach: | They propose a task of HeadLine Grouping and a dataset consisting of 20,056 pairs of news headlines, each labeled with a binary judgement as to whether the pair belongs within the same group. |
| Outcome: | The proposed model outperforms human models on a task consisting of 20,056 pairs of headlines on HLGD and a dataset with a binary judgement. |
Recurrent Attention for Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research questions the importance of dot-product self-attention in Transformer models and shows that most attention heads learn simple positional patterns. |
| Approach: | They propose a novel mechanism to replace dot-product self-attention with a recurrent atteNtion mechanism that directly learns attention weights without token-to-token interaction. |
| Outcome: | The proposed model outperforms the Transformer model on translation tasks with fewer parameters and inference time. |
Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: A Case Study on Hateful Memes (2024.lrec-main)
Copied to clipboard
| Challenge: | Causal analyses define semantics, while gradient-based methods are essential to eXplainable AI (XAI), interpreting the model’s ‘black box’. |
| Approach: | They propose to integrate causal analysis and XAI to integrate a model's mechanisms into their analysis by integrating a dataset of hateful meme detection models. |
| Outcome: | The proposed model can detect hateful memes using intersectionality principles and summarized attention scores highlight distinct behaviors of three Transformer models. |
The NLP Task Effectiveness of Long-Range Transformers (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing benchmarks on long-range attention models have not been sufficient to develop efficient Transformers and their practical application on complex NLP tasks. |
| Approach: | They propose to benchmark 7 Transformer variants on 5 difficult NLP tasks and 7 datasets to examine their capacity for long-range attention. |
| Outcome: | The proposed models have advantages on content selection and query-guided decoding, but they come with previously unrecognized drawbacks such as insufficient attention to distant tokens and accumulated approximation error. |
Graph-to-Graph Transformer for Transition-based Dependency Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for conditioning on graphs and predicting graphs are weak, but they are effective for transition-based dependency parsing. |
| Approach: | They propose a Transformer architecture for conditioning on and predicting arbitrary graphs. |
| Outcome: | The proposed architecture outperforms the state-of-the-art in transition-based dependency parsing on English Penn Treebank and 13 languages of Universal Dependencies Treebanks. |
Beyond Position: the emergence of wavelet-like properties in Transformers (2025.acl-long)
Copied to clipboard
| Challenge: | Despite its widespread adoption, theoretical limitations in positional encodings are resolved by developing emergent, wavelet-like processing strategies. |
| Approach: | They propose to use Rotary Position Embeddings to develop emergent, wavelet-like properties that compensate for the positional encoding’s theoretical limitations. |
| Outcome: | The attention heads evolve to implement multi-resolution processing analogous to wavelet transforms. |
Slender-Mamba: Fully Quantized Mamba in 1.58 Bits From Head to Toe (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved significant performance improvements in natural language processing domain, but require large computational resources for training and inference. |
| Approach: | They propose to use a language model architecture based on State-Space Models to quantify embedding and projection layers of a model with 150 B tokens from scratch. |
| Outcome: | The proposed language model architecture reduces costs by compressing context windows during inference while reducing the cost of training and inference. |
COCKATIEL: COntinuous Concept ranKed ATtribution with Interpretable ELements for explaining neural net classifiers on NLP (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent debates have shown that attention maps and attribution methods are unreliable. |
| Approach: | They propose a model-agnostic XAI technique that generates meaningful explanations from the last layer of a neural net model trained on an NLP classification task by using Non-Negative Matrix Factorization to discover concepts the model leverages to make predictions. |
| Outcome: | The proposed technique generates meaningful explanations from the last layer of a neural net model trained on an NLP classification task without compromising the accuracy of the underlying model or requiring a new one to be trained. |
Towards Explainable Evaluation of Language Models on the Semantic Similarity of Visual Concepts (2022.coling-1)
Copied to clipboard
Maria Lymperaiou, George Manoliadis, Orfeas Menis Mastromichalakis, Edmund G. Dervakos, Giorgos Stamou
| Challenge: | Recent advances in NLP research have focused on robustness and explainability issues of their evaluation strategies. |
| Approach: | They propose to use pre-trained transformers to evaluate semantic similarity for visual vocabularies . they propose to provide explainable metrics for understanding the quality of retrieved instances . |
| Outcome: | The proposed metrics highlight inabilities of widely used evaluation methods and highlight weaknesses in learned linguistic representations. |
Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evidence suggests that pre-trained Transformers encode commonsense knowledge . however, the extent to which this knowledge is acquired is unclear . |
| Approach: | They inject verbalized knowledge into pre-training minibatches and evaluate generalization . they find generalization does not improve over the course of pre- training from scratch . |
| Outcome: | The proposed model generalizes to supported inferences after pre-training on the injected knowledge. |
Combining Global Sparse Gradients with Local Gradients in Distributed Neural Network Training (D19-1)
Copied to clipboard
| Challenge: | In recent years, neural network models have grown dramatically in terms of number of parameters, so exchanging gradients during data-parallel training is costly in terms both of bandwidth and time. |
| Approach: | They propose to combine the compressed global gradient with the local gradient to restore Transformer convergence while RNNs converge faster. |
| Outcome: | The proposed method restores transformer convergence while RNNs converge faster. |
Self-Adjust Softmax (2025.emnlp-main)
Copied to clipboard
Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi, Jing Xiong, Xiaozhe Ren, Chao Huang, Zhenguo Li, Yu Li
| Challenge: | Usually, tokens with larger attention scores are important for the final prediction. |
| Approach: | They propose to modify softmax(z) to z softmax and its normalized variant to improve the Transformer attention mechanism by making minor adjustments to the softmax function. |
| Outcome: | The proposed model provides enhanced gradient properties compared to the vanilla softmax function. |
Evaluating Tokenizers Impact on OOVs Representation with Transformers Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained Transformer models have proven their effectiveness in adapting to multiple NLP tasks and domains. |
| Approach: | They evaluated three categories of out-of-vocabulary words using three French domain-specific datasets on the legal, medical, and energetical domains to robustly analyze these categories. |
| Outcome: | The proposed models can create new representations for out-of-vocabulary words by adding external morpho-syntactic context rather than improving the semantic understanding of the words directly. |
CICLe: Conformal In-Context Learning for Largescale Multi-Class Food Risk Classification (2024.findings-acl)
Copied to clipboard
| Challenge: | Contaminated or adulterated food poses a substantial risk to human health. |
| Approach: | They present a dataset of 7,546 text messages describing public food recalls. |
| Outcome: | The proposed model outperforms RoBERTa and XLM-R on classes with low support while reducing energy consumption. |
Norm of Mean Contextualized Embeddings Determines their Variance (2025.coling-main)
Copied to clipboard
| Challenge: | Contextualized embeddings vary by context, even for the same token . a recent study shows a trade-off between the norm and the variance of the embedded word . |
| Approach: | They show that contextualized embeddings vary by context, even for the same token . they focus on the norm of the mean embeddment and the variance of the embeddables . |
| Outcome: | The proposed method is efficient and efficient for embeddings in sentences. |
Rethinking the Value of Transformer Components (2020.coling-main)
Copied to clipboard
| Challenge: | Empirical results show that certain components are more important than others . we propose a new training strategy that can improve Transformer models by distinguishing unimportant components . |
| Approach: | They propose a training strategy that distinguishes the unimportant components in training . they compare the impact of individual component (sub-layer) on model performance . |
| Outcome: | The proposed training strategy can improve translation performance by distinguishing unimportant components in training. |
Powerformer: Efficient and High-Accuracy Privacy-Preserving Language Model with Homomorphic Encryption (2025.acl-long)
Copied to clipboard
| Challenge: | a new privacy-preserving language model, Powerformer, is designed to reduce computation overhead while maintaining model performance. |
| Approach: | They propose an efficient homomorphic encryption-based privacy-preserving language model . it incorporates three key techniques to optimize encrypted computations . |
| Outcome: | The proposed model achieves 45% reduction in computation time compared to state-of-the-art models . authors say the model preserves data privacy and AI capabilities in MLaaS environments . |
Beyond Noise: Mitigating the Impact of Fine-grained Semantic Divergences on Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Prior work treats all types of mismatches between source and target as noise . Consequently, it remains unclear how noisy parallel training samples impact NMT training. |
| Approach: | They propose a divergent-aware NMT framework that uses factors to help NMT recover from the degradation caused by naturally occurring divergences. |
| Outcome: | The proposed framework improves translation quality and model calibration on EN-FR tasks. |
Exploring Transformer Text Generation for Medical Dataset Augmentation (2020.lrec-1)
Copied to clipboard
| Challenge: | Natural Language Processing (NLP) is a powerful tool to unlock the vast troves of unstructured data in clinical text. |
| Approach: | They propose a method for augmenting unstructured patient information to allow NLP model development on downstream clinically relevant tasks. |
| Outcome: | The proposed method beats baselines on a downstream classification task and can be used for NLP model development. |
MATE: Multi-view Attention for Table Transformer Efficiency (2021.emnlp-main)
Copied to clipboard
| Challenge: | Tables are ubiquitous on the web, and are rich in information. |
| Approach: | They propose a sparse-attention Transformer architecture for modeling documents that contain large tables. |
| Outcome: | The proposed architecture scales linearly with respect to speed and memory, and can handle documents containing more than 8000 tokens with current accelerators. |
When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute (2021.emnlp-main)
Copied to clipboard
| Challenge: | Large language models are difficult to train because of the growing computation time and cost. |
| Approach: | They propose a highly-efficient architecture that combines fast recurrence and attention for sequence modeling. |
| Outcome: | The proposed model achieves state-of-the-art on a Wiki-103 and Billion Word datasets using 1.6 days of training on an 8-GPU machine. |
Eye-Tracking Features Masking Transformer Attention in Question-Answering Tasks (2024.lrec-main)
Copied to clipboard
| Challenge: | Eye movement features are considered to be direct signals reflecting human attention distribution with a low cost to obtain, inspiring researchers to augment language models with eye-tracking (ET) data. |
| Approach: | They select first fixation duration (FFD) and total reading time (TRT) as the cognitive signals to guide Transformer attention in question-answering tasks. |
| Outcome: | The proposed models improve but compromise stability when augmenting with ET data. |
AdapterDrop: On the Efficiency of Adapters in Transformers (2021.emnlp-main)
Copied to clipboard
Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, Iryna Gurevych
| Challenge: | Recent approaches to transformer models are expensive to fine-tune, slow for inference, and have large storage requirements. |
| Approach: | They propose a method to remove adapters from transformer layers during training and inference . they show that AdapterDrop can dynamically reduce computational overhead . |
| Outcome: | The proposed approach reduces computational overhead while maintaining performance over multiple tasks with minimal loss of performance. |
XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection (2024.findings-acl)
Copied to clipboard
| Challenge: | XMoE leverages small experts and a threshold-based router to selectively engage only essential parameters. |
| Approach: | They propose a novel MoE that leverages small experts to selectively engage only essential parameters. |
| Outcome: | The proposed model can reduce computation load at MoE layers by over 50% without sacrificing performance. |
Impact of Task Adapting on Transformer Models for Targeted Sentiment Analysis in Croatian Headlines (2024.lrec-main)
Copied to clipboard
| Challenge: | BERT models are often taken off-the-shelf and fine-tuned on a downstream task. |
| Approach: | They propose an extra stage of self-supervised task-adaptive pre-training to perform a task on a number of Croatian-supporting Transformer models. |
| Outcome: | The proposed approach improves performance across multilingual models but not in Croatian-dominant models. |
SecFormer: Fast and Accurate Privacy-Preserving Inference for Transformer Models via SMPC (2024.findings-acl)
Copied to clipboard
| Challenge: | a growing number of cloud-based inference services are relying on SMPC to protect data privacy. |
| Approach: | They propose a framework for Privacy-Preserving Inference for Transformer models that eliminates exponential and maximum operations in PPI without sacrificing model performance. |
| Outcome: | The proposed framework outperforms MPCFormer in terms of performance and efficiency . it is 3.57 and 3.58 times faster than PUMA for BERTBASE and BERTLARGE . |
Pre-training and Evaluating Transformer-based Language Models for Icelandic (2022.lrec-1)
Copied to clipboard
| Challenge: | Pre-trained models obtain state-of-the-art performance on a wide variety of NLP tasks, including Question Answering (QA), Named Entity Recognition (NER), Part-of Speech (POS) tagging and Automatic Text Summarization (ATS). |
| Approach: | They pre-train four types of monolingual ELECTRA and ConvBERT models and compare them to a previously trained monolingual RoBERTa model and multilingual mBERT model. |
| Outcome: | The models outperform a multilingual model on four downstream tasks. |
Data Factors for Better Compositional Generalization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent diagnostic datasets on compositional generalization expose severe problems . state-of-the-art models trained on larger and more general datasets show better generalization ability . |
| Approach: | They conduct an empirical analysis by training Transformer models on a variety of training sets with different data factors including dataset scale, pattern complexity, example difficulty, etc. |
| Outcome: | The proposed model training on larger datasets improves on compositional generalization tasks. |
Continuous-Time Attention: PDE-Guided Mechanisms for Long-Sequence Transformers (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to optimize attention for long sequences have been limited by their computational cost. |
| Approach: | They propose a framework that infuses partial differential equations into the Transformer’s attention mechanism to better handle long sequences. |
| Outcome: | The proposed framework achieves consistent performance gains over standard and long-sequence Transformer variants across a range of tasks. |
On the (In-)Security of the Shuffling Defense in the Transformer Secure Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Existing work reveals only randomly permuted activations to the client, allowing adversaries to extract model weights. |
| Approach: | They propose an attack that aligns differently shuffled activations to a common permutation and exploits them to extract model weights. |
| Outcome: | The proposed attack can align shuffled activations to a common permutation and exploit them to extract model weights with a query cost of approximately $1. |
Value Residual Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decoder-only transformers fail to preserve initial token-level information in deeper layers. |
| Approach: | They propose a new architecture that incorporates value residual connections in addition to hidden state residuals. |
| Outcome: | The proposed architecture reduces KV cache size by nearly half with only a small performance penalty and can be integrated with other KV-efficient methods. |